Video Super-Resolution through Image Registration based on Structural Similarity

It is commonly known that the Mean Square Error (MSE) does not accurately reflect the subjective image quality for most video enhancement tasks. Among the various image quality metrics, Structural Similarity metric provides remarkably good prediction of the subjective scores. In this paper a new registration method based on contribution of structural similarity measurement to the well known Lucas-Kanade (LK) algorithm has been proposed. The core of the proposed method is contributing the SSIM in the sum of squared difference of images along with the Levenberg-Marquardt optimization approach in LK algorithm. Mathematical derivation of the proposed method, based on the unified framework of Baker et. al.is given. The proposed registration algorithm is applied to a video super resolution problem, successfully. Various objective and subjective comparisons show the superior performance of the proposed method. Preprint submitted to Image and Vision Computing October 3, 2009 Figure 1: “sliding window” technique for video super resolution [3].